{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Increasing the Maximum Message Size for gRPC\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequistes\n",
    "You will need\n",
    " - [Git clone of Seldon Core](https://github.com/SeldonIO/seldon-core)\n",
    " - [Helm](https://github.com/kubernetes/helm)\n",
    " - [Minikube](https://github.com/kubernetes/minikube) version v0.24.0 or greater\n",
    " - [seldon-core Python package](https://pypi.org/project/seldon-core/) (```pip install seldon-core```)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create Cluster\n",
    "\n",
    "Start minikube and ensure custom resource validation is activated and there is 5G of memory. \n",
    "\n",
    "An example start command using the kvm2 driver would look like:\n",
    "```\n",
    "minikube start --vm-driver kvm2 --memory 4096 --feature-gates=CustomResourceValidation=true --extra-config=apiserver.Authorization.Mode=RBAC\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "namespace/seldon created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create namespace seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Context \"minikube\" modified.\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl config set-context $(kubectl config current-context) --namespace=seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "clusterrolebinding.rbac.authorization.k8s.io/kube-system-cluster-admin created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Install Helm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "serviceaccount/tiller created\n",
      "clusterrolebinding.rbac.authorization.k8s.io/tiller created\n",
      "$HELM_HOME has been configured at /home/janis/.helm.\n",
      "\n",
      "Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.\n",
      "\n",
      "Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.\n",
      "To prevent this, run `helm init` with the --tiller-tls-verify flag.\n",
      "For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation\n",
      "Happy Helming!\n"
     ]
    }
   ],
   "source": [
    "!kubectl -n kube-system create sa tiller\n",
    "!kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller\n",
    "!helm init --service-account tiller"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start seldon-core"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Install the custom resource definition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core-crd\n",
      "LAST DEPLOYED: Tue Dec 18 12:17:56 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/ConfigMap\n",
      "NAME                     DATA  AGE\n",
      "seldon-spartakus-config  3     1s\n",
      "\n",
      "==> v1beta1/CustomResourceDefinition\n",
      "NAME                                         AGE\n",
      "seldondeployments.machinelearning.seldon.io  1s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                        DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-spartakus-volunteer  1        0        0           0          1s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME                        SECRETS  AGE\n",
      "seldon-spartakus-volunteer  1        1s\n",
      "\n",
      "==> v1beta1/ClusterRole\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  0s\n",
      "\n",
      "==> v1beta1/ClusterRoleBinding\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  0s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "NOTES: TODO\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-core-crd --name seldon-core-crd --set usage_metrics.enabled=true"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core\n",
      "LAST DEPLOYED: Tue Dec 18 12:17:59 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/Pod(related)\n",
      "NAME                                                 READY  STATUS             RESTARTS  AGE\n",
      "seldon-core-ambassador-56cb8fc595-zj2lj              0/1    ContainerCreating  0         0s\n",
      "seldon-core-seldon-apiserver-59978fbf45-8wwzn        0/1    ContainerCreating  0         0s\n",
      "seldon-core-seldon-cluster-manager-5449b8dd5f-8xrt6  0/1    ContainerCreating  0         0s\n",
      "seldon-core-redis-567db9ddcf-779cl                   0/1    ContainerCreating  0         0s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME    SECRETS  AGE\n",
      "seldon  1        1s\n",
      "\n",
      "==> v1beta1/Role\n",
      "NAME          AGE\n",
      "seldon-local  1s\n",
      "ambassador    1s\n",
      "\n",
      "==> v1/RoleBinding\n",
      "NAME    AGE\n",
      "seldon  1s\n",
      "\n",
      "==> v1beta1/RoleBinding\n",
      "NAME        AGE\n",
      "ambassador  1s\n",
      "\n",
      "==> v1/Service\n",
      "NAME                          TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)                        AGE\n",
      "seldon-core-ambassador-admin  NodePort   10.98.239.177  <none>       8877:32647/TCP                 1s\n",
      "seldon-core-ambassador        NodePort   10.106.95.143  <none>       80:31878/TCP,443:32369/TCP     1s\n",
      "seldon-core-seldon-apiserver  NodePort   10.106.141.26  <none>       8080:31633/TCP,5000:31460/TCP  0s\n",
      "seldon-core-redis             ClusterIP  10.105.197.97  <none>       6379/TCP                       0s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                                DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-core-ambassador              1        1        1           0          0s\n",
      "seldon-core-seldon-apiserver        1        1        1           0          0s\n",
      "seldon-core-seldon-cluster-manager  1        1        1           0          0s\n",
      "seldon-core-redis                   1        1        1           0          0s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "Thank you for installing Seldon Core.\n",
      "\n",
      "Documentation can be found at https://github.com/SeldonIO/seldon-core\n",
      "\n",
      "\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-core --name seldon-core --namespace seldon \\\n",
    "    --set ambassador.enabled=true"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check all services are running before proceeding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"seldon-core-seldon-cluster-manager\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"seldon-core-seldon-cluster-manager\" successfully rolled out\n",
      "deployment \"seldon-core-seldon-apiserver\" successfully rolled out\n",
      "Waiting for deployment \"seldon-core-ambassador\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"seldon-core-ambassador\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/seldon-core-seldon-cluster-manager\n",
    "!kubectl rollout status deploy/seldon-core-seldon-apiserver\n",
    "!kubectl rollout status deploy/seldon-core-ambassador "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up REST and gRPC methods\n",
    "\n",
    "**Ensure you port forward to API Gateway**\n",
    "\n",
    "REST:\n",
    "\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n seldon -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].metadata.name}') -n seldon 8003:8080\n",
    "```\n",
    "\n",
    "GRPC:\n",
    "\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n seldon -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].metadata.name}') -n seldon 8004:5000\n",
    "```\n",
    "\n",
    "**Ensure you port forward ambassador**:\n",
    "\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n seldon -l service=ambassador -o jsonpath='{.items[0].metadata.name}') -n seldon 8005:8080\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\r\n",
      "    \u001b[34;01m\"apiVersion\"\u001b[39;49;00m: \u001b[33m\"machinelearning.seldon.io/v1alpha2\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"kind\"\u001b[39;49;00m: \u001b[33m\"SeldonDeployment\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"metadata\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "            \u001b[34;01m\"app\"\u001b[39;49;00m: \u001b[33m\"seldon\"\u001b[39;49;00m\r\n",
      "        },\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"seldon-model\"\u001b[39;49;00m\r\n",
      "    },\r\n",
      "    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"test-deployment\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"oauth_key\"\u001b[39;49;00m: \u001b[33m\"oauth-key\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"oauth_secret\"\u001b[39;49;00m: \u001b[33m\"oauth-secret\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"predictors\"\u001b[39;49;00m: [\r\n",
      "            {\r\n",
      "                \u001b[34;01m\"componentSpecs\"\u001b[39;49;00m: [{\r\n",
      "                    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "                        \u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "                            {\r\n",
      "                                \u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier:1.0\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "                                    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "                                        \u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "                                    }\r\n",
      "                                }\r\n",
      "                            }\r\n",
      "                        ],\r\n",
      "                        \u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m\r\n",
      "                    }\r\n",
      "                }],\r\n",
      "                \u001b[34;01m\"graph\"\u001b[39;49;00m: {\r\n",
      "                    \u001b[34;01m\"children\"\u001b[39;49;00m: [],\r\n",
      "                    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier\"\u001b[39;49;00m,\r\n",
      "                    \u001b[34;01m\"endpoint\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m : \u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "\t\t    },\r\n",
      "                    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"MODEL\"\u001b[39;49;00m\r\n",
      "                },\r\n",
      "                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"example\"\u001b[39;49;00m,\r\n",
      "                \u001b[34;01m\"replicas\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "\t\t    \u001b[34;01m\"version\"\u001b[39;49;00m : \u001b[33m\"v1\"\u001b[39;49;00m\r\n",
      "\t\t}\r\n",
      "            }\r\n",
      "        ]\r\n",
      "    }\r\n",
      "}\r\n"
     ]
    }
   ],
   "source": [
    "!pygmentize resources/model.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create Seldon Deployment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Deploy the runtime graph to kubernetes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/seldon-model created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/model.json -n seldon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the status of the SeldonDeployment. **When ready the replicasAvailable should be 1**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[state:Available predictorStatus:[map[name:test-deployment-example-7cd068f replicas:1 replicasAvailable:1]]]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-model -o jsonpath='{.status}' -n seldon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get predictions - no grpx max message size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_utils import *\n",
    "API_GATEWAY_REST=\"localhost:8003\"\n",
    "API_GATEWAY_GRPC=\"localhost:8004\"\n",
    "API_AMBASSADOR=\"localhost:8005\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Send a small request which should suceed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"2ac70653-8b36-46f6-b5e8-611c3c090029\",\"token_type\":\"bearer\",\"expires_in\":43199,\"scope\":\"read write\"}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"uqb9mt5qi3790bvvbgmus5oklr\"\n",
       "  requestPath {\n",
       "    key: \"classifier\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.0892753869230776\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_api_gateway('oauth-key','oauth-secret',None,API_GATEWAY_REST,API_GATEWAY_GRPC,10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test via Ambassador"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"6nlf542sb7mgvukpsot2l361bh\"\n",
       "  requestPath {\n",
       "    key: \"classifier\"\n",
       "    value: \"seldonio/mock_classifier:1.0\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.08619452296855779\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_ambassador(\"seldon-model\",None,API_AMBASSADOR,data_size=10,rows=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Send a large request which will be above the default gRPC message size and will fail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"2ac70653-8b36-46f6-b5e8-611c3c090029\",\"token_type\":\"bearer\",\"expires_in\":43187,\"scope\":\"read write\"}\n"
     ]
    },
    {
     "ename": "_Rendezvous",
     "evalue": "<_Rendezvous of RPC that terminated with:\n\tstatus = StatusCode.CANCELLED\n\tdetails = \"Received RST_STREAM with error code 8\"\n\tdebug_error_string = \"{\"created\":\"@1545135751.395194140\",\"description\":\"Error received from peer\",\"file\":\"src/core/lib/surface/call.cc\",\"file_line\":1095,\"grpc_message\":\"Received RST_STREAM with error code 8\",\"grpc_status\":1}\"\n>",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31m_Rendezvous\u001b[0m                               Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-17-0371fba8d063>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mgrpc_request_api_gateway\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'oauth-key'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'oauth-secret'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mAPI_GATEWAY_REST\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mAPI_GATEWAY_GRPC\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1000000\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/work/GIT/fork-seldon-core/notebooks/seldon_utils.py\u001b[0m in \u001b[0;36mgrpc_request_api_gateway\u001b[0;34m(oauth_key, oauth_secret, namespace, rest_endpoint, grpc_endpoint, data_size, rows, data)\u001b[0m\n\u001b[1;32m     57\u001b[0m     \u001b[0mstub\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mprediction_pb2_grpc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSeldonStub\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mchannel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     58\u001b[0m     \u001b[0mmetadata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'oauth_token'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtoken\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 59\u001b[0;31m     \u001b[0mresponse\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mstub\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mPredict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrequest\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mrequest\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mmetadata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmetadata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     60\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mresponse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     61\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.conda/envs/seldon-core/lib/python3.6/site-packages/grpc/_channel.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, request, timeout, metadata, credentials)\u001b[0m\n\u001b[1;32m    512\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrequest\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmetadata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcredentials\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    513\u001b[0m         \u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcall\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_blocking\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrequest\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmetadata\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcredentials\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 514\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0m_end_unary_response_blocking\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcall\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    515\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    516\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mwith_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrequest\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmetadata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcredentials\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.conda/envs/seldon-core/lib/python3.6/site-packages/grpc/_channel.py\u001b[0m in \u001b[0;36m_end_unary_response_blocking\u001b[0;34m(state, call, with_call, deadline)\u001b[0m\n\u001b[1;32m    446\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mstate\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mresponse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    447\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 448\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0m_Rendezvous\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdeadline\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    449\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    450\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31m_Rendezvous\u001b[0m: <_Rendezvous of RPC that terminated with:\n\tstatus = StatusCode.CANCELLED\n\tdetails = \"Received RST_STREAM with error code 8\"\n\tdebug_error_string = \"{\"created\":\"@1545135751.395194140\",\"description\":\"Error received from peer\",\"file\":\"src/core/lib/surface/call.cc\",\"file_line\":1095,\"grpc_message\":\"Received RST_STREAM with error code 8\",\"grpc_status\":1}\"\n>"
     ]
    }
   ],
   "source": [
    "grpc_request_api_gateway('oauth-key','oauth-secret',None,API_GATEWAY_REST,API_GATEWAY_GRPC,1000000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "ename": "_Rendezvous",
     "evalue": "<_Rendezvous of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"upstream connect error or disconnect/reset before headers\"\n\tdebug_error_string = \"{\"created\":\"@1545135755.480635396\",\"description\":\"Error received from peer\",\"file\":\"src/core/lib/surface/call.cc\",\"file_line\":1095,\"grpc_message\":\"upstream connect error or disconnect/reset before headers\",\"grpc_status\":14}\"\n>",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31m_Rendezvous\u001b[0m                               Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-18-a412d4b14655>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mgrpc_request_ambassador\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"seldon-model\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mAPI_AMBASSADOR\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mdata_size\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1000000\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mrows\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/work/GIT/fork-seldon-core/notebooks/seldon_utils.py\u001b[0m in \u001b[0;36mgrpc_request_ambassador\u001b[0;34m(deploymentName, namespace, endpoint, data_size, rows, data)\u001b[0m\n\u001b[1;32m    115\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    116\u001b[0m         \u001b[0mmetadata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'seldon'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mdeploymentName\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'namespace'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mnamespace\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 117\u001b[0;31m     \u001b[0mresponse\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mstub\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mPredict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrequest\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mrequest\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mmetadata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmetadata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    118\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mresponse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    119\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.conda/envs/seldon-core/lib/python3.6/site-packages/grpc/_channel.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, request, timeout, metadata, credentials)\u001b[0m\n\u001b[1;32m    512\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrequest\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmetadata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcredentials\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    513\u001b[0m         \u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcall\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_blocking\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrequest\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmetadata\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcredentials\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 514\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0m_end_unary_response_blocking\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcall\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    515\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    516\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mwith_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrequest\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmetadata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcredentials\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.conda/envs/seldon-core/lib/python3.6/site-packages/grpc/_channel.py\u001b[0m in \u001b[0;36m_end_unary_response_blocking\u001b[0;34m(state, call, with_call, deadline)\u001b[0m\n\u001b[1;32m    446\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mstate\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mresponse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    447\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 448\u001b[0;31m         \u001b[0;32mraise\u001b[0m \u001b[0m_Rendezvous\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstate\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdeadline\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    449\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    450\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31m_Rendezvous\u001b[0m: <_Rendezvous of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"upstream connect error or disconnect/reset before headers\"\n\tdebug_error_string = \"{\"created\":\"@1545135755.480635396\",\"description\":\"Error received from peer\",\"file\":\"src/core/lib/surface/call.cc\",\"file_line\":1095,\"grpc_message\":\"upstream connect error or disconnect/reset before headers\",\"grpc_status\":14}\"\n>"
     ]
    }
   ],
   "source": [
    "grpc_request_ambassador(\"seldon-model\",None,API_AMBASSADOR,data_size=1000000,rows=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"seldon-model\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/model.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "release \"seldon-core\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!helm delete seldon-core --purge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Allowing larger gRPC messages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Recreate seldon-core with extra annotation for the API Gateway"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core\n",
      "LAST DEPLOYED: Tue Dec 18 12:22:05 2018\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1beta1/Deployment\n",
      "NAME                                DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-core-ambassador              1        1        1           0          0s\n",
      "seldon-core-seldon-apiserver        1        1        1           0          0s\n",
      "seldon-core-seldon-cluster-manager  1        1        1           0          0s\n",
      "seldon-core-redis                   1        1        1           0          0s\n",
      "\n",
      "==> v1/Pod(related)\n",
      "NAME                                                 READY  STATUS             RESTARTS  AGE\n",
      "seldon-core-ambassador-56cb8fc595-495hc              0/1    ContainerCreating  0         0s\n",
      "seldon-core-ambassador-56cb8fc595-zj2lj              1/1    Terminating        0         4m5s\n",
      "seldon-core-seldon-apiserver-59978fbf45-8wwzn        0/1    Terminating        0         4m5s\n",
      "seldon-core-seldon-apiserver-64d4fb745f-sqgsp        0/1    ContainerCreating  0         0s\n",
      "seldon-core-seldon-cluster-manager-5449b8dd5f-8xrt6  0/1    Terminating        0         4m5s\n",
      "seldon-core-seldon-cluster-manager-5449b8dd5f-qbxvr  0/1    ContainerCreating  0         0s\n",
      "seldon-core-redis-567db9ddcf-779cl                   0/1    Terminating        0         4m5s\n",
      "seldon-core-redis-567db9ddcf-8rq74                   0/1    ContainerCreating  0         0s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME    SECRETS  AGE\n",
      "seldon  1        0s\n",
      "\n",
      "==> v1beta1/Role\n",
      "NAME          AGE\n",
      "seldon-local  0s\n",
      "ambassador    0s\n",
      "\n",
      "==> v1/RoleBinding\n",
      "NAME    AGE\n",
      "seldon  0s\n",
      "\n",
      "==> v1beta1/RoleBinding\n",
      "NAME        AGE\n",
      "ambassador  0s\n",
      "\n",
      "==> v1/Service\n",
      "NAME                          TYPE       CLUSTER-IP      EXTERNAL-IP  PORT(S)                        AGE\n",
      "seldon-core-ambassador        NodePort   10.99.125.114   <none>       80:30477/TCP,443:30761/TCP     0s\n",
      "seldon-core-ambassador-admin  NodePort   10.105.153.197  <none>       8877:31224/TCP                 0s\n",
      "seldon-core-seldon-apiserver  NodePort   10.98.196.218   <none>       8080:31106/TCP,5000:31384/TCP  0s\n",
      "seldon-core-redis             ClusterIP  10.99.9.251     <none>       6379/TCP                       0s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "Thank you for installing Seldon Core.\n",
      "\n",
      "Documentation can be found at https://github.com/SeldonIO/seldon-core\n",
      "\n",
      "\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-core --name seldon-core --namespace seldon \\\n",
    "    --set ambassador.enabled=true \\\n",
    "    -f grpc_size_values.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wait for seldon core deployment to be running"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment \"seldon-core-seldon-cluster-manager\" successfully rolled out\n",
      "deployment \"seldon-core-seldon-apiserver\" successfully rolled out\n",
      "Waiting for deployment \"seldon-core-ambassador\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"seldon-core-ambassador\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/seldon-core-seldon-cluster-manager\n",
    "!kubectl rollout status deploy/seldon-core-seldon-apiserver\n",
    "!kubectl rollout status deploy/seldon-core-ambassador "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we change our SeldonDeployment to include a annotation for max grpx message size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\r\n",
      "    \u001b[34;01m\"apiVersion\"\u001b[39;49;00m: \u001b[33m\"machinelearning.seldon.io/v1alpha2\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"kind\"\u001b[39;49;00m: \u001b[33m\"SeldonDeployment\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"metadata\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "            \u001b[34;01m\"app\"\u001b[39;49;00m: \u001b[33m\"seldon\"\u001b[39;49;00m\r\n",
      "        },\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"seldon-model\"\u001b[39;49;00m\r\n",
      "    },\r\n",
      "    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"annotations\"\u001b[39;49;00m: {\r\n",
      "\t    \u001b[34;01m\"seldon.io/grpc-max-message-size\"\u001b[39;49;00m:\u001b[33m\"10000000\"\u001b[39;49;00m\r\n",
      "        },\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"test-deployment\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"oauth_key\"\u001b[39;49;00m: \u001b[33m\"oauth-key\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"oauth_secret\"\u001b[39;49;00m: \u001b[33m\"oauth-secret\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"predictors\"\u001b[39;49;00m: [\r\n",
      "            {\r\n",
      "                \u001b[34;01m\"componentSpecs\"\u001b[39;49;00m: [{\r\n",
      "                    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "                        \u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "                            {\r\n",
      "                                \u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/mock_classifier_grpc:1.0\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "                                    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "                                        \u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "                                    }\r\n",
      "                                }\r\n",
      "                            }\r\n",
      "                        ],\r\n",
      "                        \u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m20\u001b[39;49;00m\r\n",
      "                    }\r\n",
      "                }],\r\n",
      "                \u001b[34;01m\"graph\"\u001b[39;49;00m: {\r\n",
      "                    \u001b[34;01m\"children\"\u001b[39;49;00m: [],\r\n",
      "                    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"classifier\"\u001b[39;49;00m,\r\n",
      "                    \u001b[34;01m\"endpoint\"\u001b[39;49;00m: {\r\n",
      "\t\t\t\u001b[34;01m\"type\"\u001b[39;49;00m : \u001b[33m\"GRPC\"\u001b[39;49;00m\r\n",
      "\t\t    },\r\n",
      "                    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"MODEL\"\u001b[39;49;00m\r\n",
      "                },\r\n",
      "                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"grpc-size\"\u001b[39;49;00m,\r\n",
      "                \u001b[34;01m\"replicas\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"annotations\"\u001b[39;49;00m: {\r\n",
      "\t\t    \u001b[34;01m\"predictor_version\"\u001b[39;49;00m : \u001b[33m\"v1\"\u001b[39;49;00m\r\n",
      "\t\t}\r\n",
      "            }\r\n",
      "        ]\r\n",
      "    }\r\n",
      "}\r\n"
     ]
    }
   ],
   "source": [
    "!pygmentize resources/model_grpc_size.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/seldon-model created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create -f resources/model_grpc_size.json -n seldon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **Make sure you rerun the port forwards above as a new seldon core API gateway and Ambassador will have started**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[predictorStatus:[map[replicasAvailable:0 name:test-deployment-grpc-size-fd60a01 replicas:1]] state:Creating]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-model -o jsonpath='{.status}' -n seldon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Send a large message. This time it should succeed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"500e9e17-a1f1-4174-a14c-cf988f673eb4\",\"token_type\":\"bearer\",\"expires_in\":43125,\"scope\":\"read write\"}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"3k8cfubi9vcoogsup3fk1itdna\"\n",
       "  requestPath {\n",
       "    key: \"classifier\"\n",
       "    value: \"seldonio/mock_classifier_grpc:1.0\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.08192989817777528\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_api_gateway('oauth-key','oauth-secret',None,API_GATEWAY_REST,API_GATEWAY_GRPC,data_size=1000000,rows=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Send a request via ambassador. This should also succeed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "meta {\n",
       "  puid: \"9sir96lidq9gees0bb70s1uqdb\"\n",
       "  requestPath {\n",
       "    key: \"classifier\"\n",
       "    value: \"seldonio/mock_classifier_grpc:1.0\"\n",
       "  }\n",
       "}\n",
       "data {\n",
       "  names: \"proba\"\n",
       "  tensor {\n",
       "    shape: 1\n",
       "    shape: 1\n",
       "    values: 0.08191949643488104\n",
       "  }\n",
       "}"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grpc_request_ambassador(\"seldon-model\",None,API_AMBASSADOR,data_size=1000000,rows=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
